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ABSTRACT 



This paper describes a qualitative study that examined 
students ' use of higher-order thinking as they use an interactive multimedia 
program based on a situated learning framework. The learning environment 
incorporated the following characteristics of a situated learning model : an 
authentic context; multiple perspectives; expert performances; coaching and 
scaffolding; opportunities for collaboration, reflection and articulation; 
and authentic assessment. The multimedia program focused on the issue of 
assessment and presented a number of resources for preservice teachers of 
mathematics to investigate from a variety of perspectives. As they used the 
program, the preservice teachers were given a complex and sustained authentic 
task to investigate. It was hypothesized that the situated learning model 
would lead to higher- order thinking in its implementation. Four groups of two 
students were videotaped using the resource over 2 weeks, and their 
discussion was transcribed for analysis with qualitative analysis software. 
Analysis showed that the majority of thinking was higher order; social, 
procedural, and lower-order talk was present in reduced proportions. Six 
figures present a flow chart of a classification scheme of student talk and 
graphs of proportions of categories of talk and of higher-order thinking. A 
table contains a summary chart of classification of student talk. (DLS) 
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Abstract 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



This paper describes a qualitative study into students’ use of higher-order thinking 
as they use an interactive multimedia program based on a situated learning 
framework. The analysis of types of talk used by students as they worked with the 
program clearly shows that the majority of their thinking was higher order, as 
defined by [Resnick 1987] and other theorists. Social, procedural and lower-order 
talk was less evident but present in their talk in reduced proportions. These 
findings suggest that a multimedia program based on a situated learning approach 
can provide a learning environment capable of supporting and maintaining 
substantial levels of higher-order thinking. 
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Introduction 

Many educators and researchers discuss the importance of engaging students in activities which require more 
than the simple application of rules and procedures. [Collins, Brown and Newman 1989] contend that few 
educational resources (including interactive multimedia) are devoted to higher-order problem solving 
activities, and few activities require students to use cognitive and metacognitive strategies and processes. 
[Collins, et al. 1989] suggest that higher order learning — ‘cognitive and metacognitive strategies and 
processes’ — can ‘best be taught’ through methods that employ a situated learning approach (p. 455). 

While higher-order thinking might most simply be described as ‘all intellectual tasks that call for more than 
information retrieval’ [Baker 1990], [Lewis and Smith 1993] give a more comprehensive definition: ‘Higher- 
order thinking occurs when a person takes new information and information stored in memory and interrelates 
and/or rearranges and extends this information to achieve a purpose or find possible answers in perplexing 
situations’ (p. 136). Many studies have produced methods and procedures to classify and define higher-order 
learning. However, as [Newmann 1990] points out, each approach has its own persuasive rationale. He 
contends that it is not productive to try to choose the best, but more sensible ‘to search for a common 
conception that embraces diverse emphases but which attracts professional consensus’ (p. 42). 
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Situated Learning 

To date, there appears to have been scant research into whether higher order thinking is enhanced and 
promoted by learning environments based on a situated learning framework. The majority of studies that have 
been conducted to investigate students’ use of higher order thinking as they use multimedia packages report 
little evidence of in mainstream student activity (e.g., [Frampton 1994]. While the proponents of situated 
learning continue to claim that higher order learning is a consequence of learning within a situated learning 
environment, very little research has been done to evaluate the impact of situated learning elements on 
students’ thinking, particularly with regard to the use of interactive multimedia programs. 

This paper describes a study which investigated students’ thinking as they used an interactive multimedia 
program based on the situated learning approach. The learning environment incorporated nine characteristics 
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of a situated learning model, namely: an authentic context ; complex authentic activities ; multiple perspectives ; 
expert performances ; coaching and scaffolding ; opportunities for collaboration , reflection and articulation ; 
and authentic assessment [Herrington & Oliver 1995]; [Herrington, Sparrow, Herrington, & Oliver 1997]. The 
learning environment was designed for preservice teachers of mathematics. The multimedia program focused 
on the issue of assessment and presented a number of resources for preservice teachers to investigate from a 
variety of perspectives, such as short video clips of assessment strategies being used in classrooms, and 
interviews with teachers and students on the strategies; and a variety of text documents, such as a description 
of each strategy and mathematics experts’ views. As they used the program, the preservice teachers were given 
a complex and sustained authentic task to investigate in the form of two letters: one from a parent complaining 
to the school about the number of tests her child is required to take, and the other a memo to the new teachers 
at the school (the preservice teachers) asking them to prepare a new assessment plan for mathematics in the 
school. It was hypothesised that the situated learning model, used in the design of the program, would lead to 
substantial levels of higher-order thinking in its implementation. Four groups of two students were videotaped 
using the resource over two semester weeks, and their discussion was transcribed for analysis with qualitative 
analysis software. 



Framework for analysis 

Several frameworks have been developed for analysis of student cognition within learning environments, 
which served as a useful starting point for the current study (e.g.,[Marland, Patching, & Putt 1992]; [Alexander 
& Frampton 1994]; [Nastasi & Clements 1992] . These frameworks use the basis of student talk as a means of 
identifying cognition and thinking with the nature of the talk being used as an indicator of the form of 
cognition and thinking undertaken by the student. The work of [Henri 1992] was most useful in providing a 
model for analysis of the data in the study. Henri developed her framework for analysis of student talk in a 
computer-mediated conferencing environment. Student exchanges during lessons were monitored and analysed 
using content analysis. Clearly, the categorisation of spoken messages within learning environments is a 
practised format for analysis of student talk. 

In order to classify students’ talk as they used the interactive multimedia program on assessment, a table of 
indicators was prepared based on the characterisation of [Resnick 1987] of higher-order thinking. To simplify 
the classification for the purpose of the research, several of Resnick’s characterisations were combined to 
enable more distinct categories to be drawn. It was also necessary to draw up similar criteria for the 
classification of talk which could not be considered higher order — Social, Procedural and Lower order — rather 
than have a simple category of Non-higher order thinking. All student talk was classified according to the 
scheme which is represented diagrammatically below [Fig. 1]: 




Figure 1: Flow chart of classification scheme of student talk 






The unit of analysis 



In order to assign student talk to a category, it was necessary to define the ‘grain size’ of the unit of speech to 
be classified. The method used was to count each instance of a type of talk as it occurred [Henri 1992] which 
enabled the detection of types of talk which may have been neglected by other methods, such as the counting 
of passages of dialogue, turns at talk, or individual words. Each category of talk, together with a definition, a 
short summary and example of type is given in [Tab. 1], 



Table 1: Summary chart of classification of student talk 



Category 


Sub-category 


Definition 


Example of type 


Social 


Off-task 


Any statement not related to the subject matter 


G: We got caught in a traffic jam ... We didn' t see it 

happen but we saw a van, the side of it was all 
smashed in. 




On-task 


Any social statements which relate in some way to 
the task 


C: Hey 1 know this guy ... He was my teacher. What a 

spin. 1 wonder what he is doing. 


Procedural 


Equipment 


Any exchange of information related to the 
equipment (such as the operation of the computer, 
monitor, keyboard, CD-ROM drive etc.) 


R: No you won' t have a volume on this ... it' d be on 

the Apple menu. You have to go into systems folder 
then control panel. 




Software 


Any exchange of information related to the software 
(any functional aspect of the program itself) 


C: That didn' t work. 

R: Did you push copy first? 




Task 


Any exchange of information related to the task (the 
formal requirements of the oral or written report). 


G: Are we actually supposed to prepare this as a report 

to staff? 

D: No, 1 don' t think so. 1 think we prepare our notes and 

then just say it. 


Lower order 




Any student talk which is routine, requiring little 
thought, or the mechanical application of well known 
rules. 


C: You are missing an r in strategies. 

R; They are playing dominoes. It is a little primary 

school class. 


Higher order 


Uncertainty 


Any student talk which involved deciding on an 
approach to adopt, suggesting a course of action, or 
any expression of dilemma or uncertainty. 


L: So really we want to look at all of them don' t we? 

E: OK do you want to start putting anything into our 

notebooks? 




Path of action 


Any talk which involved decisions about which 
elements of the program to access, what to save in 
the notebook and negotiations on how to proceed. 


R: We should go right through the whole lot again. We 

need to make more notes on it. 




Judgement 


Any statement or question which referred to 
students' attempts to interpret and defend their 
understanding of the issues presented in the 
assessment program. 


G: If you' re going to do anecdotal records ... 

D: You wouldn' t carry them around. You' d make notes 

and put them in later. 

G: That' s the whole point of having them combined. 




Multiple 

perspectives 


A statement or question was classified in this 
category if it suggested an alternative approach or 
challenged a conclusion, or previously made point, by 
providing an alternative perspective. 


R: OK investigations, factual ... Ah, factual, factual 

recall, rote learning. 

C: No, it' s not saying you learn everything by rote, it' s 

just saying that sometimes you want students to 
respond automatically.. 




Imposing 

meaning 


Talk was classified into this category if it referred to 
a possible solution to a problem or suggested altern- 
ative solution, if it expressed a decision about what 
to believe, or the creation of a new idea, or if it drew 
cautious conclusions. 


G: Obviously we can introduce all of the strategies but 

not to start with. 1 think in the lower primary you 
probably can' t expect them to do a mathematical 
investigation, so if they start out with oral work in 
the lower school and work towards doing the written 
ones in the upper school. 




Metacognition 


Any comments which showed that students were 
aware of their own thinking and performance, and 
comments related to the use of this awareness to 
improve performance. 


G: What was that? I didn' t get any of that. 

E: 1 don' t know what to do. Where is that piece of 

paper 1 had before? [The activity]. 



Discussion 



The analysis of the transcripts showed that higher-order thinking was a substantial component in all the 
students’ talk. In each of the four groups observed, the proportion of higher-order thinking to the other major 
categories observed was quite consistent and was measured at around 70% of the total talk. 
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[Fig. 2] shows the proportion of categories for Group 1. The two students in this group used a substantial 
proportion of higher-order thinking in their talk as they used the multimedia program. Lower order comments, 
together with social talk, were kept to a minimum and procedural matters occupied only a moderate amount of 
their time. Like Group 1, the students in Group 2 used a substantial proportion of higher-order talk (70%), a 
moderate amount of procedural and minimal lower order talk. Of all the groups, this group had the most social 
talk, largely centred around discussion of their mutual friends, computers and work from other units of study. 
The students in Group 3 were least typical in their pattern of talk. There was a complete absence of social talk 
which appeared to be the result of the fact that they did not know each other prior to commencing work on the 
program. The amount of higher-order talk was 71%. A relatively high amount of procedural talk was observed 
in Group 3 and this was largely related to recurring computer equipment problems which were not 
satisfactorily rectified until the second week of the study. Like the other groups, however, they had a high 
proportion of higher-order talk, and minimal use of lower order talk. Group 4 were similar to the other groups 
in their use of a substantial amount of higher-order talk (68%). The remaining categories of talk were also 
comparable with other groups. 

Proportions of higher-order thinking 

The high level of higher-order thinking amongst all the groups meant that there was a substantial number of 
units of meaning in students’ talk which could be classified according to the classification scheme (Table 1). 
While the amount of talk was quite consistent between the groups, there were many differences in the extent 
and levels of the various forms of higher-order thinking evident in the talk. The proportion of each of these 
types to the whole of higher-order talk for Group 1 is shown in [Fig. 3]. 
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Figure 3: Proportion of categories of higher-order thinking: Group 1 

The students in Group 1 used a substantial amount of all types of higher-order talk identified in the 
classification scheme. As with most of the groups, Uncertainty , Path of action and Judgement comprised the 
major part of their talk, with the other classifications making up the remainder. Clearly, these students were 
relatively comfortable working together, with the total of Uncertainty and Path of action totalling less than 
half their talk. The moderate proportion of Multiple perspectives appeared to indicate that these students were 
not excessively argumentative or critical, but were not afraid to challenge each other or the program when they 
saw the need. These students were also capable of spending a good proportion of their time Imposing meaning 
on their learning and coming to conclusions about the task and the recommendations to include in their 
reports. As with all the groups, these students’ expression of metacognitive awareness was minimal, and it is 
possible that this is a type of thinking which does not manifest in the spoken word as well as the other 
categories of talk. 
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Figure 4: Proportion of categories of higher-order thinking: Group 2 
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The interesting aspect of the analysis talk in Group 2 [Fig. 4] is the relatively low proportion of Uncertainty 
and Path of action compared to Judgement and Multiple perspectives. This division indicates that these 
students were forthright and confident in working out their path through the interactive multimedia program, 
and that they appeared to very comfortable working together. The high proportion of Multiple perspectives 
indicates that they adopted a very critical approach to the information they were obtaining from the program 
and from each other. The high proportion of Imposing meaning also seems to indicate that they were then 
readily able to consolidate the information into a meaningful form. 
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16% Multiple 




Figure 5: Proportion of categories of higher-order thinking: Group 3 

While it is difficult to nominate an optimum spread of talk to categories within higher-order thinking, clearly 
Group 3 had difficulties which become apparent with closer scrutiny of their types of talk. The high 
percentage of time spent in both Path of action and Uncertainty reflects the tentative nature of their 
collaboration [Fig 5]. Almost three quarters of the total talk fell into one or other of these categories. As they 
were unaccustomed to working together, the students appeared to spend a relatively large proportion of their 
time consulting with each other about the nature of their collaboration — how they were to proceed, how to 
interact, and the responsibility each was to take in the process. Another interesting finding in the proportion of 
talk for this group was the almost complete absence of Multiple perspectives talk between the two students. 
Their collaboration was characterised by a reluctance to challenge each other’s ideas or to challenge the 
perspectives that were presented in the multimedia program. 



The students in Group 4 [Fig 6] used relatively little talk which was classified as Uncertainty and Path of 
action indicating that they needed minimal talk to establish a working relationship and a proposed plan of 
action. Like others, they demonstrated a high proportion of Multiple perspectives as they used the program. 
However, a large proportion of this talk was an argumentative style of interaction they used as they worked 
together, rather than a thoughtful disagreement with ideas presented in the program. This is possibly evident in 
the fact that there was a minimal proportion of talk which was classified in the category of Imposing meaning. 
These students, unlike Group 2, did not use the multiple perspectives they offered each other to inform the 
meaning of the task. 
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Figure 6: Proportion of categories of higher-order thinking: Group 4 



Discussion 

It is interesting to note the wide disparity between types of higher-order thinking used by the students as they 
used the multimedia program. The findings show that all the students used a substantial proportion of higher- 
order thinking in the situated learning environment, where other studies (e.g. [Frampton 1994]; [Oliver & 
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McLoughlin 1996] have shown little. The possibility exists that the classification scheme developed to analyse 
students’ talk was not a precise enough instrument to truly reflect the cognition of students as they used the 
program. For example, comments such as ‘What do you want to do now?’ may be closer to a cliche or 
automatic response than a thoughtful reflection of the best course of action. However, even with a 
reclassification of Uncertainty and Path of action as Lower order thinking, Higher-order thinking remains a 
high proportion of the type of talk used by all the groups. An explanation for the amount of this type of talk is 
that the constructivist nature of the learning environment provided greater opportunities for students’ higher- 
order thinking. 

Another interesting finding was the non-sequential nature of the types of thinking used by the students, 
confirming the contentions of [Resnick 1987] and [Newmann 1990] that higher-order thinking is relative and 
non-hierarchical, and counter to behavioural theorists such as [Bloom 1956] and to some extent, [Gagn6 
1985], where progression to each level of the hierarchy is dependent upon mastery of the previous level. If one 
accepted a hierarchical approach to classification of thinking, it might be expected that students would begin 
with a little social talk to establish their working relationship; then procedural talk as they worked out the 
computer equipment, the software and the task; they might then be expected to move to lower order talk 
before using higher-order talk later in the session. Interestingly, there was no sequence or pattern to their use 
of talk. From the beginning, the students moved freely and without notice to any type of talk. 

The analysis of types of talk used by students as they worked with the interactive multimedia program clearly 
shows that the majority of their thinking was higher order, as defined by [Resnick 1987] and other theorists. 
Social, procedural and lower-order talk was less evident but present in their talk in reduced proportions. These 
findings confirmed our expectations that a multimedia program based on a situated learning approach could 
provide a learning environment capable of supporting and maintaining substantial levels of higher-order 
thinking. Our findings contrast with many previous studies exploring students’ cognition and thinking in 
multimedia use. The findings suggest that the instructional design embraced in the situated learning program 
and its implementation in this study successfully integrated a number of elements and components often 
missing in applications and uses of multimedia in contemporary learning settings. 
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